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Abstract — Backpressure-based adaptive routing algorithms 
where each packet is routed along a possibly different path 
have been extensively studied in the literature. However, such 
algorithms typically result in poor delay performance and involve 
high implementation complexity. In this paper, we develop a 
new adaptive routing algorithm built upon the widely-studied 
back-pressure algorithm. We decouple the routing and scheduling 
components of the algorithm by designing a probabilistic routing 
table which is used to route packets to per-destination queues. 
The scheduling decisions in the case of wireless networks are 
made using counters called shadow queues. The results are 
also extended to the case of networks which employ simple 
forms of network coding. In that case, our algorithm provides a 
low-complexity solution to optimally exploit the routing-coding 
tradeoff. 



I, Introduction 

The back-pressure algorithm introduced in [25 1 has been 
widely studied in the literature. While the ideas behind 
scheduling using the weights suggested in that paper have been 
successful in practice in base stations and routers, the adaptive 
routing algorithm is rarely used. The main reason for this is 
that the routing algorithm can lead to poor delay performance 
due to routing loops. Additionally the implementation of the 
back-pressure algorithm requires each node to maintain per- 
destination queues which can be burdensome for a wireline 
or wireless router. Motivated by these considerations, we re- 
examine the back-pressure routing algorithm in the paper and 
design a new algorithm which has much superior performance 
and low implementation complexity. 

Prior work in this area ll22ll has recognized the importance 
of doing shortest-path routing to improve delay performance 
and modified the back-pressure algorithm to bias it towards 
taking shortest-hop routes. A part of our algorithm has similar 
motivating ideas, but we do much more. In addition to prov- 
ably throughput-optimal routing which minimizes the number 
of hops taken by packets in the network, we decouple routing 
and scheduling in the network through the use of probabilistic 
routing tables and the so-called shadow queues. The min-hop 
routing idea was studied first in a conference paper [7| and 
shadow queues were introduced in |6|, but the key step of 
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decoupling the routing and scheduling which leads to both 
dramatic delay reduction and the use of per-next-hop queueing 
is original here. The min-hop routing idea is also studied in 
[26] but their solution requires even more queues than the 
original back-pressure algorithm. 

We also consider networks where simple forms of network 
coding is allowed fl7l . In such networks, a relay between two 
other nodes XORs packets and broadcast them to decrease the 
number of transmissions. There is a tradeoff between choosing 
long routes to possibly increase network coding opportunities 
(see the notion of reverse carpooling in |10|) and choosing 
short routes to reduce resource usage. Our adaptive routing 
algorithm can be modified to automatically realize this tradeoff 
with good delay performance. In addition, network coding 
requires each node to maintain more queues lfl31 and our 
routing solution at least reduces the number of queues to be 
maintained for routing purposes, thus partially mitigating the 
problem. An offline algorithm for optimally computing the 
routing-coding tradeoff was proposed in 11231 . Our optimiza- 
tion formulation bears similarities to this work but our main 
focus is on designing low-delay on-line algorithms. Back- 
pressure solutions to network coding problems have also been 
studied in ||14| . ifTTI . (8), but the adaptive routing -coding 
tradeoff solution that we propose here has not been studied 
previously. 

We summarize our main results below. 

• Using the concept of shadow queues, we decouple routing 
and scheduling. A shadow network is used to update a 
probabilistic routing table which packets use upon arrival 
at a node. The back-pressure-based scheduling algorithm 
is used to serve FIFO queues over each link. 

• The routing algorithm is designed to minimize the av- 
erage number of hops used by packets in the network. 
This idea, along with the scheduling/routing decoupling, 
leads to delay reduction compared with the traditional 
back-pressure algorithm. 

• Each node has to maintain counters, called shadow 
queues, per destination. This is very similar to the idea of 
maintaining a routing table per destination. But the real 
queues at each node are per-next-hop queues in the case 
of networks which do not employ network coding. When 
network coding is employed, per-previous-hop queues 
may also be necessary but this is a requirement imposed 
by network coding, not by our algorithm. 

• The algorithm can be applied to wireline and wireless 
networks. Extensive simulations show dramatic improve- 
ment in delay performance compared to the back-pressure 
algorithm. 



The rest of the paper is organized as follows. We present 
the network model in Section [TTJ In Section [TiT] and [TV] the tra- 
ditional back-pressure algorithm and its modified version are 
introduced. We develop our adaptive routing and scheduling 
algorithm for wireline and wireless networks with and without 



network coding in Section |V}|VI| and VII In Section VIII the 
simulation results are presented. We conclude our paper in 
Section |IX] 

II. The Network Model 

We consider a multi-hop wireline or wireless network 
represented by a directed graph Q = (Af,C), where J\f is the 
set of nodes and C is the set of directed links. A directed link 
that can transmit packets from node n to node j is denoted 
by (nj) £ C. We assume that time is slotted and define the 
link capacity c n j to be the maximum number of packets that 
link (nj) can transmit in one time slot. 

Let T be the set of flows that share the network. Each 
flow is associated with a source node and a destination node, 
but no route is specified between these nodes. This means 
that the route can be quite different for packets of the same 
flow. Let b(f) and e(f) be source and destination nodes, 
respectively, of flow /. Let Xf be the rate (packets/slot) at 
which packets are generated by flow /. If the demand on 
the network, i.e., the set of flow rates, can be satisfied by 
the available capacity, there must exist a routing algorithm 
and a scheduling algorithm such that the link rates lie in 
the capacity region. To precisely state this condition, we 
define /if to be the rate allocated on link (nj) to packets 
destined for node d. Thus, the total rate allocated to all flows 
at link (nj) is given by /i n j := \~] Mnj- Clearly, for the 

deAf 

network to be able to meet the traffic demand, we should have: 



{^n]}(nj)eC G A, 

where A is the capacity region of the network for 1-hop traffic. 
The capacity region of the network for 1-hop traffic contains 
all sets of rates that are stabilizable by some kind of scheduling 
policy assuming all traffics are 1-hop traffic. As a special case, 
in the wireline network, the constraints are: 

As opposed to A, let T denote the capacity region of the multi- 
hop network, i.e., for any set of flows {x/}/ e jr S T, there 
exists some routing and scheduling algorithms that stabilize 
the network. 

In addition, a flow conservation constraint must be satisfied 
at each node, i.e., the total rate at which traffic can possibly 
arrive at each node destined to d must be less than or equal to 
the total rate at which traffic can depart from the node destined 
to d : 



^r 



E 



/*f„ 



(1) 



2^ X f I {b{f)=nAJ)=d} 

< E <r 

j:(nj)£C 

where / denotes the indicator function. Given a set of arrival 
rates x — {xj}f^j- that can be accommodated by the network, 



one version of the multi-commodity flow problem is to find 
the traffic splits /xf such that M is satisfied. However, finding 
the appropriate traffic split is computationally prohibitive and 
requires knowledge of the arrival rates. The back-pressure 
algorithm to be described next is an adaptive solution to the 
multi-commodity flow problem. 

III. Throughput-Optimal Back-pressure 
Algorithm and Its Limitations 

The back-pressure algorithm was first described in ll25ll 
in the context of wireless networks and independently dis- 
covered later in Q as a low-complexity solution to certain 
multi-commodity flow problems. This algorithm combines 
the scheduling and routing functions together. While many 
variations of this basic algorithm have been studied, they 
primarily focus on maximizing throughput and do not consider 
QoS performance. Our algorithm uses some of these ideas 
as building blocks and therefore, we first describe the basic 
algorithm, its drawbacks and some prior solutions. 

The algorithm maintains a queue for each destination at each 
node. Since the number of destinations can be as large as the 
number of nodes, this per-destination queueing requirement 
can be quite large for practical implementation in a network. 
At each link, the algorithm assigns a weight to each possible 
destination which is called back-pressure. Define the back- 
pressure at link (nj) for destination d at slot t to be 

w njM = Qnd[t] ~ Qjd[t], 

where Q n d[t] denotes the number of packets at node n destined 
for node d at the beginning of time slot t. Under this notation, 
Qnn[t] = 0,Vt. Assign a weight w n j to each link (nj), where 
w n j is defined to be the maximum back-pressure over all 
possible destinations, i.e., 

w n j[t] = maxw n ,[t]. 

d J 

Let d* ■ be the destination which has the maximum weight on 
link (nj), 



d *nM = argmax{wf At}}. 



(2) 



If there are ties in the weights, they can be broken arbitrarily. 
Packets belonging to destination d n At} are scheduled for 
transmission over the activated link (nj). A schedule is a set of 
links that can be activated simultaneously without interfering 
with each other. Let T denote the set of all schedules. The 
back-pressure algorithm finds an optimal schedule ir* [t} which 
is derived from the optimization problem: 



7r* [t] = arg max ^ c nj w nj [t] . 



(3) 

Specially, if the capacity of every link has the same value, 
the chosen schedule maximizes the sum of weights in any 
schedule. 

At time t, for each activated link (nj) G tt* [t] we remove 
c n j packets from Q n d* [t] if possible, and transmit those 
packets to Qjd* m . We assume that the departures occur first 
in a time slot, and external arrivals and packets transmitted 
over a link (nj) in a particular time slot are available to node 



j at the next time slot. Thus the evolution of the queue Q n d [t] 
is as follows: 

Qnd[t + ]] = Qnd[t] - 2^/ ^{<**^ [*]=<*} Ani [*] 

j:(nj)eC 

+ 22 1 {d' n [t\=d} hn[A 
l:(ln)£C 

+ E 1 {b(f)=n,e(f)=d} «/[*]. 



(4) 



where jl n j [t] is the number of packets transmitted over link 
(nj) in time slot t and a,f[t] is the number of packets generated 
by flow / at time t. It has been shown in [25 1 that the back- 
pressure algorithm maximizes the throughput of the network. 

A key feature of the back-pressure algorithm is that packets 
may not be transferred over a link unless the back-pressure 
over a link is non-negative and the link is included in the 
picked schedule. This feature prevents further congesting 
nodes that are already congested, thus providing the adaptivity 
of the algorithm. Notice that because all links can be activated 
without interfering with each other in the wireline network, T 
is the set of all links. Thus the back-pressure algorithm can be 
localized at each node and operated in a distributed manner 
in the wireline network. 

The back-pressure algorithm has several disadvantages that 
prohibit practical implementation: 

• The back-pressure algorithm requires maintaining queues 
for each potential destination at each node. This queue 
management requirement could be a prohibitive overhead 
for a large network. 

• The back-pressure algorithm is an adaptive routing algo- 
rithm which explores the network resources and adapts 
to different levels of traffic intensity. However it might 
also lead to high delays because it may choose long paths 
unnecessarily. High delays are also a result of maintaining 
a large number of queues at each node. Only one queue 
can be scheduled at a time, and the unused service could 
further contribute to high latency. 

In this paper, we address the high delay and queueing 
complexity issues. The computational complexity issue for 
wireless networks is not addressed here. We simply use the 
recently studied greedy maximal scheduling (GMS) algorithm. 
Here we call it the largest-weight-first algorithm, in short, 
LWF algorithm. LWF algorithm requires the same queue struc- 
ture that the back-pressure algorithm uses. It also calculates the 
back-pressure at each link using the same way. The difference 
between these two algorithms only lies in the methods to pick a 
schedule. Let S denote the set of all links initially. Let J\fb(l) be 
the set of links within the interference range of link I including 
I itself. At each time slot, the LWF algorithm picks a link I 
with the maximum weight first, and removes links within the 
interference range of link I from S, i.e., S = S\Afb{l)', then it 
picks the link with the maximum weight in the updated set S, 
and so forth. It should be noticed that LWF algorithm reduces 
the computational complexity with a price of the reduction 
of the network capacity region. The LWF algorithm where 
the weights are queue lengths (not back-pressures) has been 
extensively studied in g), El, 0, ED, H9). While these 



studies indicate that there may be reduction in throughput 
due to LWF in certain special network topologies, it seems 
to perform well in simulations and so we adopt it here. 

In the rest of the paper, we present our main results which 
eliminate many of the problems associated with the back- 
pressure algorithm. 

iv. mln-resource routing using back-pressure 
Algorithm 



As mentioned in Section III the back-pressure algorithm 



explores all paths in the network and as a result may choose 
paths which are unnecessarily long which may even contain 
loops, thus leading to poor performance. We address this 
problem by introducing a cost function which measures the 
total amount of resources used by all flows in the network. 
Specially, we add up traffic loads on all links in the network 
and use this as our cost function. The goal then is to minimize 
this cost subject to network capacity constraints. 

Given a set of packet arrival rates that lie within the capacity 
region, our goal is to find the routes for flows so that we use as 
few resources as possible in the network. Thus, we formulate 
the following optimization problem: 

min ^2 Vnj (5) 

S.t. Y^ XfI{b(f)=nMf)=d} + E tfn < E ^™J ' 

/e.F (hi)ec (nj)ec 

Vd e AT, n G AT, 

{Hnj}(nj)eC G A. 

We now show how a modification of the back-pressure 
algorithm can be used to solve this min-resource routing 
problem. (Note that similar approaches have been used in 
El, ED, ED, EZ), E3 to solve related resource allocation 
problems.) 

Let {q n d} be the Lagrange multipliers corresponding to the 
flow conservation constraints in problem (T5]). Appending these 
constraints to the objective, we get 

£^S E Pmi +H*"i(H X / / {«=K/),e(/)=d} 
^ (nj)eC n,d f£T 



o 



+ E **£.- E 

(ln)eC (nj)eC 



(6) 



(nj)eC d 

- E qnd E X f I {n=b(f),e(f)=d}) ■ 
n,d feF 

If the Lagrange multipliers are known, then the optimal //, can 
be found by solving 

max } VnjWnj 

where w n j = ma,x(q n d — qjd — !)• The form of the constraints 

d 

in (pb suggests the following update algorithm to compute 



q n d 



q n d[t + 1] = qnd[t] + Tf(J2 x f I - 



{n=b(f),e(f)=d} 



fe? 



E 



/';, 



(nj)£C 



o 



(7) 



where jj is a step-size parameter. Notice that Mq n d[t] looks 
very much like a queue update equation, except for the fact 
that arrivals into Q n d from other links may be smaller than 
/j.f n when Qid does not have enough packets. This suggests 
the following algorithm. 

Min-resource routing by back-pressure: At time slot t, 
• Each node n maintains a separate queue of packets for 

each destination d; its length is denoted Q n d[t\- Each link 

is assigned a weight 



U-'n 



\t] = max 
d 



Qnd[t] 



1 

M 



Qjd[t] - 1 



(8) 



where M > is a parameter. 
Scheduling/routing rule: 



n*[t] e argmax Y] c nj w nj [t]. (9) 

• For each activated link (nj) £ ir* [t] we remove c n j 
packets from Q n d* \t\ if possible, and transmit those 
packets to Qjd* .[t], where d* [t] achieves the maximum 
in ([8). 

Note that the above algorithm does not change if we replace 
the weights in dSl by the following, re-scaled ones: 



>j n j[t) = max(Q nd [£] 

d 



Q jd [t]-M), 



(10) 



and therefore, compared with the traditional back-pressure 
scheduling/routing, the only difference is that each link weight 
is equal to the maximum differential backlog minus parameter 
M. (M = reverts the algorithm to the traditional one.) For 
simplicity, we call this algorithm M -back-pressure algorithm. 
The performance of the stationary process which is "pro- 
duced" by the algorithm with fixed parameter M is within 
o(l) of the optimal as M goes to oo (analogous to the proofs 
in EH, d; see also the related proof in 1(121, lull): 



E 



(n 3 )ec 






0(1) 



where /i* is an optimal solution to d5), 

Although M -back-pressure algorithm could reduce the de- 
lay by forcing flows to go through shorter routes, simula- 
tions indicate a significant problem with the basic algorithm 
presented above. A link can be scheduled only if the back- 
pressure of at least one destination is greater than or equal to 
M. Thus, at light to moderate traffic loads, the delays could 
be high since the back-pressure may not build up sufficiently 
fast. In order to overcome all these adverse issues, we develop 
a new routing algorithm in the following section. The solution 
also simplifies the queueing data structure to be maintained at 
each node. 



V. PARN: Packet- by- Packet Adaptive Routing and 
Scheduling Algorithm For Networks 

In this section, we present our adaptive routing and schedul- 
ing algorithm. We will call it PARN (Packet-by-Packet Adap- 
tive Routing for Networks) for ease for repeated reference 
later. First, we introduce the queue structure that is used in 
PARN. 

In the traditional back-pressure algorithm, each node n has 
to maintain a queue q n d for each destination d. Let \M\ and 
\T>\ denote the number of nodes and the number of destinations 
in the network, respectively. Each node maintains \T>\ queues. 
Generally, each pair of nodes can communicate along a path 
connecting them. Thus, the number of queues maintained at 
each node can be as high as one less than the number of nodes 
in the network, i.e., |2?|=|jV| — 1. 

Instead of keeping a queue for every destination, each node 
n maintains a queue q n j for every neighbor j, which is called 
a real queue. Notice that real queues are per-neighbor queues. 
Let J n denote the number of neighbors of node n, and let 
J max = max„ J n . The number of queues at each node is no 
greater than J max . Generally, J max is much smaller than \J\f\ . 
Thus, the number of queues at each node is much smaller 
compared with the case using the traditional back-pressure 
algorithm. 

In additional to real queues, each node n also maintains a 
counter, which is called shadow queue, p n d for each desti- 
nation d. Unlike the real queues, counters are much easier 
to maintain even if the number of counters at each node 
grows linearly with the size of the network. A back-pressure 
algorithm run on the shadow queues is used to decide which 
links to activate. The statistics of the link activation are further 
used to route packets to the per-next-hop neighbor queues 
mentioned earlier. The details are explained next. 



A. Shadow Queue Algorithm - M -back-pressure Algorithm 

The shadow queues are updated based on the movement 
of fictitious entities called shadow packets in the network. 
The movement of the fictitious packets can be thought of as 
an exchange of control messages for the purposes of routing 
and schedule. Just like real packets, shadow packets arrive 
from outside the network and eventually exit the network. The 
external shadow packet arrivals are general as follows: when 
an exogenous packet arrives at node n to the destination d, 
the shadow queue p n d is incremented by 1, and is further 
incremented by 1 with probability e in addition. Thus, if the 
arrival rate of a flow / is Xf , then the flow generates "shadow 
traffic" at a rate Xf(l + e). In words, the incoming shadow 
traffic in the network is (1 + e) times of the incoming real 
traffic. 

The back-pressure for destination d on link (nj) is taken to 
be 

wi j [t]=Pnd[t]-Pjd{t}-M, 

where M is a properly chosen parameter. The choice of M 
will be discussed in the simulations section. 



The evolution of the shadow queue p n d [t] is 

Pnd[t + 1] = Pnd[t]~ Yl I {dl ] [t\=d}Vn ] [t] 
j:(nj)eC 

+ H hdtJt]=d} fan[t] (U) 

l:(ln)eC 

+ Y 1 {b(f)=n,e(f)=d} a,f[t], 

where jj, n j [t] is the number of shadow packets transmitted over 
link (nj) in time slot t, d* ■[£] is the destination that has the 
maximum weight on link (nj), and a,f[t] is the number of 
shadow packets generated by flow / at time t. The number of 
shadow packets scheduled over the links at each time instant 
is determined by the back-pressure algorithm in equation d5). 

From the above description, it should be clear that the 
shadow algorithm is the same as the traditional back-pressure 
algorithm, except that it operates on the shadow queueing 
system with an arrival rate slightly larger than the real external 
arrival rate of packets. Note the shadow queues do not involve 
any queueing data structure at each node; there are no packets 
to maintain in a FIFO order in each queue. The shadow queue 
is simply a counter which is incremented by 1 upon a shadow 
packet arrival and decremented by 1 upon a departure. 

The back-pressure algorithm run on the shadow queues is 
used to activate the links. In other words, if 7T* ■ = 1 in d9V 
then link (nj) is activated and packets are served from the 
real queue at the link in a first-in, first-out fashion. This is, 
of course, very different from the traditional back-pressure 
algorithm where a link is activated to serve packets to a 
particular destination. Thus, we have to develop a routing 
scheme that assigns packets arriving to a node to a particular 
next-hop neighbor so that the system remains stable. We 
design such an algorithm next. 

B. Adaptive Routing Algorithms 

Now we discuss how a packet is routed once it arrives 
at a node. Let us define a variable cr^At] to be the number 
of shadow packets "transferred" from node n to node j for 
destination d during time slot t by the shadow queue algorithm. 
Let us denote by cr„- the expected value of cr^ [£], when 
the shadow queueing process is in a stationary regime; let 
a„At] denote an estimate of a„ 4 , calculated at time t. (In the 
simulations we use the exponential averaging, as specified in 
the next section.) 

At each time slot, the following sequence of operations 
occurs at each node n. A packet arriving at node n for 
destination d is inserted in the real queue q n j for next-hop 
neighbor j with probability 



PnM = 



"J 



[t] 



£ 



k:(nk)eC u nk 



aiu[t] 



(12) 



Thus, the estimates cr^At] are used to perform routing op- 
erations: in today's routers, based on the destination of a 
packet, a packet is routed to its next hop based on routing 
table entries. Instead, here, the <j's are used to probabilistically 
choose the next hop for a packet. Packets waiting at link (nj) 




Fig. 1. Probabilistic splitting algorithm in Node n. 



are transmitted over the link when that link is scheduled (See 
Figure [TV 

The first question that one must ask about the above 
algorithm is whether it is stable if the packet arrival rates 
from flows are within the capacity region of the multi-hop 
network. This is a difficult question, in general. Since the 
shadow queues are positive recurrent, "good" estimates <r{L- [t] 
can be maintained by simple averaging (e.g. as specified in 
the next section), and therefore the probabilities in ( p"2| ) will 
stay close to their "ideal" values 



pd _ 
nj 



"J 



L~dk:(n 



(nfe)££ a nk 



The following theorem asserts that the real queues are stable 

d „_ Q f,„~A „t T>d 



if PSj are fixed at P nj 



pd 
nj- 



Assume that there 



Theorem 1: Suppose, P^Jt] 
exists a delta such that {xf(l + e + S)} lies in T. Let a/[i] 
be the number of packets arriving from flow / at time slot 
t, with E(af[t]) = Xf and E(af[t]) < oo. Assume that the 
arrival process is independent across time slots and flows (this 
assumption can be considerably relaxed). Then, the Markov 
chain, jointly describing the evolution of shadow queues and 
real FIFO queues (whose state include the destination of the 
real packet in each position of each FIFO queue), is positive 
recurrent. 

Proof: The key ideas behind the proof are outlined. The 
details are similar to the proof in [5 1 and are omitted. 

• The average rate at which packets arrive to link (nj) is 
strictly smaller than the capacity allocated to the link by 
the shadow process if e > 0, (This fact is verified in 
Appendix [A] ) 

• It follows that the fluid limit of the real-queue process is 
same as that of the networks in Q. Such fluid limit is 
stable [3 1, which implies the stability of our process as 
well. 



VI. Implementation Details 

The algorithm presented in the previous section ensures that 
the queue lengths are stable. In this section, we discuss a 
number of enhancements to the basic algorithm to improve 
performance. 



A. Exponential Averaging 

To compute cr^ [t] we use the following iterative exponential 
averaging algorithm: 

a% [t] = (1-13) a d nj [t-l}+(3 a% [t] , (13) 

where < ft < 1. 

B. Token Bucket Algorithm 

Computing the average shadow rate cr„ At] and generating 
random numbers for routing packets may impose a computa- 
tional overhead of routers which should be avoided if possible. 
Thus, as an alternative, we suggest the following simple 
algorithm. At each node n, for each next-hop neighbor j and 
each destination d, maintain a token bucket r„_-. Consider the 
shadow traffic as a guidance of the real traffic, with tokens 
removed as shadow packets traverse the link. In detail, the 
token bucket is decremented by a d , [t] in each time slot, but 
cannot go below the lower bound 0: 

ri 3 [t] = max{r^[t - 1] - <[t], 0}. 

When r d nj [t - 1] - a d nj [t] < 0, we say that a^ [t] - r^ [t - 1] 
tokens (associated with bucket r„,-) are "wasted" in slot t. 
Upon a packet arrival at node n for destination d, find the 
token bucket r„,-, which has the smallest number of tokens 
(the minimization is over next-hop neighbors j), breaking ties 
arbitrarily, add the packet to the corresponding real queue q n j* 
and add one token to the corresponding bucket: 



[t]=r d [t-l] + l. 



(14) 
the 



To explain how this algorithm works, denote by cr„ 
average value of c^ ■ [t] (in stationary regime), and by i] d the 
average rate at which real packets for destination d arrive at 
node n. Due to the fact that real traffic is injected by each 
source at the rate strictly less than the shadow traffic, we have 



rt< 



E* 



d 



(15) 



For a single-node network, ( 15 i just means that arrival rate is 



less than available capacity. More generally, it is an assump- 
tion that needs to be proved. However, here our goal is to 
provide an intuition behind the token bucket algorithm, so we 



simply assume (15 1. Condition (15 1 guarantees that the token 
processes are stable (that is, roughly, they cannot runaway to 
infinity) since the total arrival rate to the token buckets at a 
node is less than the total service rate and the arrivals employ 
a join-the-shortest-queue discipline. Moreover, since r£.-[t] are 
random processes, the token buckets will "hit 0" in a non-zero 
fraction of time slots, except in some degenerate cases; this in 
turn means that the arrival rate of packets at the token bucket 
must be less than the token generation rate: 
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(16) 



where 77^ ■ is the actual rate at which packets arriving at n and 



destined for d are routed along link (nj). Inequality ( 16 1 thus 
describes the idea of the algorithm. 



Ideally, in addition to ( 16 1, we would like to have the ratios 
Vni/^tj to t> e equal across all j, i.e., the real packet arrival 



rates at the outgoing links of a node should be proportional 
to the shadow service rates. It is not difficult to see that if e 
is very small, the proportion will be close to ideal. In general, 
the token-based algorithm does not guarantee that, that is why 
it is an approximation. 



Also, to ensure implementation correctness, instead of ( 14 1, 
we use 

rf llt [t]=mm{r d Jt [t-l] + l,B}, (17) 

i.e., the value of j™,- [t] is not allowed to go above some 
relatively large value B, which is a parameter of the order 
of 0(1/ e). Under "normal circumstances", »"{,,•[£] "hitting" 
ceiling B is a rare event, occurring due to the process 
randomness. The main purpose of having the upper bound B 
is to detect serious anomalies when, for whatever reason, the 
condition ( 15 1 "breaks" for prolonged periods of time - such 
situation is detected when any r d , [t] hits the upper bound B 
frequently. 

C. Extra Link Activation 

Under the shadow back-pressure algorithm, only links with 
back-pressure greater than or equal to M can be activated. 
The stability theory ensures that this is sufficient to render 
the real queues. On the other hand, the delay performance 
can still be unacceptable. Recall that the parameter M was 
introduced to discourage the use of unnecessarily long paths. 
However, under light and moderate traffic loads, the shadow 
back-pressure at a link may be frequently less than M, and 
thus, packets at such links may have to wait a long time before 
they are processed. One way to remedy the situation is to 
activate additional links beyond those activated by the shadow 
back-pressure algorithm. 

The basic idea is as follows: in each time slot, first run the 
shadow back-pressure algorithm. Then, add additional links to 
make the schedule maximal. If the extra activation procedure 
depends only on the state of shadow queues (but beyond that, 
can be random and/or arbitrarily complex), then the stability 
result of Theorem [T] still holds (with essentially same proof). 
Informally, the stability prevails, because the shadow algorithm 
alone provides sufficient average throughput on each link, and 
adding extra capacity "does not hurt"; thus, with such extra 
activation, a certain degree of "decoupling" between routing 
(totally controlled by shadow queues) and scheduling (also 
controlled by shadow queues, but not completely) is achieved. 

For example, in the case of wireline networks, by the above 
arguments, all links can be activated all the time. The shadow 
routing algorithm ensures that the arrival rate at each link is 
less than its capacity. In this case the complete decoupling of 
routing and scheduling occurs. 

In practice, activating extra links which have large queue 
backlogs leads to better performance than activating an ar- 
bitrary set of extra links. However, in this case, the extra 
activation procedure depends on the state of real queues which 
makes the issue of validity of an analog of Theorem [T] much 
more subtle. We believe that the argument in this subsection 
provides a good motivation for our algorithm, which is con- 
firmed by simulations. 



D. The Choice of the Parameter e 

From basic queueing theory, we expect the delay at each 
link to be inversely proportional to the mean capacity minus 
the arrival rate at the link. In a wireless network, the capacity 
at a link is determined by the shadow scheduling algorithm. 
This capacity is guaranteed to be at least equal to the shadow 
arrival rate. The arrival rate of real packets is of course smaller. 
Thus, the difference between the link capacity and arrival 
rate could be proportional to epsilon. Thus, epsilon should 
be sufficiently large to ensure small delays while it should 
be sufficiently small to ensure that the capacity region is not 
diminished significantly. In our simulations, we found that 
choosing e = 0.1 provides a good tradeoff between delay 
and network throughput. 

In the case of wireline networks, recall from the previous 
subsection that all links are activated. Therefore, the parameter 
epsilon plays no role here. 

VII. Extension to The Network Coding Case 

In this section, we extend our approach to consider net- 
works where network coding is used to improve throughput. 
We consider a simple form of network coding illustrated in 
Figure [2] When i and j each have a packet to send to the 
other through an intermediate relay n, traditional transmission 
requires the following set of transmissions: send a packet a 
from i to n, then n to j, followed by j to n and n to 
i. Instead, using network coding, one can first send from i 
to n, then j to n, XOR the two packets and broadcast the 
XORed packet from n to both i and j. This form of network 
coding reduces the number of transmissions from four to three. 
However, the network coding can only improve throughput 
only if such coding opportunities are available in the network. 
Routing plays an important role in determining whether such 
opportunities exist. In this section, we design an algorithm to 
automatically find the right tradeoff between using possibly 
long routes to provide network coding opportunities and the 
delay incurred by using long routes. 
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Fig. 2. Network coding opportunity. 



A. System Model 

We still consider the wireless network represented by the 
graph Q = (Af, C). Let Xf be the rate (packets/slot) at which 
packets are generated by flow /. To facilitate network coding, 
each node must not only keep track of the destination of the 
packet, but also remember the node from which a packet was 
received. Let [if. be the rate at which packets received from 
either node I or flow /, destined for node d, are scheduled over 
link (nj). Note that, for compactness of notation, we allow I 
in the definition of [if ■ to denote either a flow or a node. We 
assume u?_, is zero when such a transmission is not feasible, 

III J 



i.e., when n is not the source node or d is not the destination 
node of flow I, or if (In) or (nj) is not in C. At node n, 
the network coding scheme may generate a coded packet by 
"XORing" two packets received from previous-hop nodes / 
and j destined for the destination nodes d and d' respectively, 
and broadcast the coded packet to nodes j and I, Let [l\ ., 
denote the rate at which coded packets can be transferred 
from node n to nodes j and I destined for nodes d and 
d 1 ', respectively. Notice that, due to symmetry, the following 



equality holds [x 



d,d' 

AH 



u .'... Assume a \ ., to be zero if at 



»iw 



n\jl 



least one of (nl), (In), (nj) and (jn) doesn't belong to C. 



Note that [if- = when d — I or d = n, and [i, - l = when 
d = n or d' = n. 

There are two kinds of transmissions in our network model: 
point-to-point transmissions and broadcast transmissions. The 
total point-to-point rate at which packets received externally 
or from a previous-hop node are scheduled on link (nj) and 
destined to d is denoted by 



u d 

r"nj ,pp 



E rim + E 



t'l, 



tier 



i-.ieAf 



and the total broadcast rate at which packets scheduled on link 
(nj) destined to d is denoted by 



r^nj^broad 



EV^ d,d' 



d< l-.ljtj 

The total point-to-point rate on link (nj) is denoted by 



E- 



Hnj,pp — 2_^ ^nj,pp 
d 



and the total broadcast rate at which packets are broadcast 
from node n to nodes j and / is denoted by 



Vn\jl ~EE^n 



d,d' 



d' d 



Let [i be the set of rates including all point-to-point transmis- 
sions and broadcast transmissions, i.e., 



M 



ll^nj.pp i{nj)i iMnlJZj (n\jl)J ■ 



The multi-hop traffic should also satisfy the flow conserva- 
tion constraints. 

Flow conservation constraints: For each node n, each neigh- 
bor j, and each destination d, we have 



l X nj,pp + ^nj,broad ^ Z_^ ^njk + 2^i Z-^i ^j'ffen' ^^ 



h d' k:k^n 



where the left-hand side denotes the total incoming traffic rate 
at link nj destined to d, and the right-hand side denotes the 
total outgoing traffic rate from link nj destined to d. For each 
node n and each destination d, we have 



E x f I wf)=n,e(f)=d} < E E i4 

where / denotes the indicator function. 



(19) 



B. Links and Schedules 

We allow broadcast transmission in our network model. 
In order to define a schedule, we first define two kinds of 
"links:" the point-to-point link and the broadcast link. A 
point-to-point link (nj) is a link that supports point-to-point 
transmission, where (nj) E C; A broadcast link (n\lj) is 
a "link" which contains links (nl) and (nj) and supports 
broadcast transmission. Let B denote the set of all broadcast 
links, thus (n\lj) G B. Let £ be the union of the set of the 
point-to-point links C and the set of the broadcast links B, 
i.e., C = CUB. 

We let r' denote the set of links that can be activated 
simultaneously. By abusing notation, r" can be thought of as 
a set of vectors where each vector is a list of l's or O's where 
a 1 corresponds to an active link and a corresponds to an 
inactive link. Then, the capacity region of the network for 1- 
hop traffic is the convex hull of all schedules, i.e., A' = co(r'). 
Thus, fie A'. 

C. Queue Structure and Shadow Queue Algorithm 

Each node n maintains a set of counters, which are called 
shadow queues, pi n d for each previous hop I and each desti- 
nation d, and pond f° r external flows destined for d at node n. 
Each node n also maintains a real queue, denoted by qi n j , for 
each previous hop I and each next-hop neighbor j, and q 0n j 
for external flows with their next hop j. 

By solving the optimization problem with flow conservation 
constraints, we can work out the back-pressure algorithm for 
network coding case (see the brief description in Appendix 
|B) , More specifically, for each link (nj) G L in the network 
and for each destination d, define the back-pressure at every 
slot to be 
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[t] 



l:(ln)eC or 1=0 

where wf nj [t] = pi n d[t] - Pnjd[t] 
[t] = arg max w" 



and 
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l:{ln)£C or 1=0 



Inj 



■M, 



(20) 



For each broadcast at node n to nodes j and I destined for d 
and d' , respectively, define the back-pressure at every slot to 
be 



«#$[*]=<,•[*] + «&[*]• 



(21) 



The weights associated with each point-to-point link (nj) G C 
and each broadcast link (n\jl) are defined as follows 



w nj [t] =max{w% j [t]}, 
w nW [t] = max{wj*jt]}, 
with d* nj [t] = argmax{w^[t]}, 
{d,d'}* n]j i[t]=^S^{w%[t]}. 

The rate vector jl* [t] at each time slot is chosen to satisfy 
p,* [t] G arg max I V] l~i n j. PP w nj [t] 

+ Y fin\jlWn\jl[t]y 
(n\Jl)&B 



(22) 



By running the shadow queue algorithm in network coding 
case, we get a set of activated links in C at each slot. 

Next we describe the evolution of the shadow queue lengths 
in the network. Notice that the shadow queues at each node n 
are distinguished by their previous hop I and their destination 
d, so pi n d only accepts the packets from previous hop I 
with destination d. The similar rule should be followed when 
packets are drained from the shadow queue pi n d- We assume 
the departures occur before arrivals at each slot, and the 
evolution of queues is given by 
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d'GAf jeN 
/ , Afej7i[*]^{fe=if n ,d=(ir„} 



(23) 



+ Yl Yl ih\LW{{d,d'}={dA'r l]nk } 

keM d'EAf 
+ Y d 'f^ 1 {b(f)=rhe(f)=d,l=0}, 

where fifi n [t] is the actual number of shadow packets sched- 
uled over link (In) and destined for d from the shadow queue 
Pkid at slot t, Aji' n fcM i s tne actual number of coded shadow 
packets transfered from node I to nodes n and k destined for 
nodes d and d! at slot t, and <if denotes the actual number 
of shadow packets from external flow / received at node n 
destined for d. 



D. Implementation Details 

The implementation details of the joint adaptive routing and 
coding algorithm are similar to the case with adaptive routing 
only, but the notation is more cumbersome. We briefly describe 
it here. 

1) Probabilistic Splitting Algorithm: The probabilistic 
splitting algorithm chooses the next hop of the packet based on 
the probabilistic routing table. Let P^Jt] be the probability 
of choosing node j as the next hop once a packet destined 
for d receives at node n from previous hop / or from external 
flows, i.e., I = at slot t. Assume that Pf n M = if (nj) <£ C. 



Obviously, EieAT^tiM 



1. Let <rj*,-[t] denote the number 



of potential shadow packets "transferred" from node n to 
node j destined for d whose previous hop is / during time 
slot t. Notice that the packet comes from an external flow if 
1 = 0. Also notice that erf- [t] is contributed by shadow traffic 
point-to-point transmission as well as shadow traffic broadcast 
transmission, i.e., 
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We keep track of the the average value of af„ [t] across time 
by using the following updating process: 



O] = 0- - ^ & Ut - 1] + /3<T? ni [i], 



(24) 



where < fl < 1. The splitting probability Pj^At} is expressed 



as follows: 
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2) Token Bucket Algorithm: At each node n, for each 
previous-hop neighbor I, next-hop neighbor j and each desti- 
nation d, we maintain a token bucket rf-. At each time slot 
t, the token bucket is decremented by oy n .,[£], but cannot go 
below the lower bound : 



Inj I 



When r{L[t- 1] -of, 



Inj I 



[*]=max{rf n Ji-l] -<,[*], 0} 



[t] < 0, we say afM 
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[t- 

1] tokens (associated with bucket rf ) are "wasted" in slot 
t. Upon a packet arrival from previous hop I at node n for 
destination d at slot t, we find the token bucket 



Inj* 



which 



has the smallest number of tokens (the minimization is over 
next-hop neighbors j), breaking ties arbitrarily, add the packet 
to the corresponding real queue qi n j* , and add one token from 
the corresponding bucket: 



.[*]=<,■•[*] + !• 



Inj' 



E. Extra link Activation 

Like the case without network coding, extra link activation 
can reduce delays significantly. As in the case without network 
coding, we add additional links to the schedule based on the 
queue lengths at each link. For extra link activation purposes, 
we only consider point-to-point links and not broadcast. Thus, 
we schedule additional point-to-point links by giving priority 
to those links with larger queue backlogs. 

VIII. Simulations 

We consider two types of networks in our simulations: 
wireline and wireless. Next, we describe the topologies and 
simulation parameters used in our simulations, and then 
present our simulation results. 

A. Simulation Settings 

1) Wireline Setting: The network shown in Figure [3] has 31 
nodes and represents the GMPLS network topology of North 
America Q~|. Each link is assume to be able to transmit 1 
packets in each slot. We assume that the arrival process is a 
Poisson process with parameter A, and we consider the arrivals 
come within a slot are considered for service at the beginning 
of the next slot. Once a packet arrives from an external flow 
at a node n, the destination is decided by probability mass 
function P n d,d — 1,2, ...N, where P n d is the probability 
that a packet is received externally at node n destined for d. 
Obviously, Y Jd , d ^ n P n d = 1, and P nn = 0. The probability 
P n d is calculated by 

Jd + Jn 

J2 ( J k + J n) ' 

k:k^n 

where J n denotes the number of neighbors of node n. Thus, 
we use P n d to split the incoming traffic to each destination 
based on the degrees of the source and the destination. 
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Fig. 3. Sprint GMPLS network topology of North America with 31 nodes. [T] 



2) Wireless Setting: We generated a random network with 
30 nodes which resulted in the topology in Figure |4] We used 
the following procedure to generate the random network: 30 
nodes are placed uniformly at random in a unit square; then 
starting with a zero transmission range, the transmission range 
was increased till the network was connected. We assume that 
each link can transmit one packet per time slot. We assume 
a 2-hop interference model in our simulations. By a fc-hop 
interference model, we mean a wireless network where a 
link activation silences all other links which are k hops from 
the activated link. The packet arrival processes are generated 
using the same method as in the wireline case. We simulate 
two cases given the network topology: the no coding case 
and the network coding case. In both wireline and wireless 
simulations, we chose j3 in ( p"3] l to be 0.02. 




Fig. 4. Wireless network topology with 30 nodes. 



B. Simulation Results 

1) Wireline Networks: First, we compare the performance 
of three algorithms: the traditional back-pressure algorithm, 
the basic shadow queue routing/scheduling algorithm without 
the extra link activation enhancement and PARN. Without 
extra link activation, to ensure that the real arrival rate at each 
link is less than the link capacity provided by the shadow 
algorithm, we choose e — 0.02. Figure B] shows delay as a 
function of the arrival rate lambda for the three algorithms. 
As can be seen from the figure, simply using a value of 



10 



M > does not help to reduce delays without extra link 
activation. The reason is that, while M > encourages the 
use of shortest paths, links with back-pressure less than M 
will not be scheduled and thus can contribute to additional 
delays. 

Next, we study the impact of M on the performance on 
PARN. 



In Figure [7] we study wireless networks without network 
coding. From the figure, we see that the delay performance 
is relatively insensitive to the choice of M as long as it 
is sufficiently greater than zero. The use of M ensures that 
unnecessary resource wastage does not occur, and thus, extra 
link activation can be used to decrease delays significantly. 



Traditional Backpressure Algorithm 
No Extra Link Activation M=2 
PARN M=2 
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Fig. 5. The impact of the parameter M in Sprint GMPLS network topology. 

Figure [6] shows the delay performance for various M with 
extra link activation in the wireline network. The delays for 
different values of M (except M — 0) are almost the same 
in the light traffic region. Once M is sufficiently larger than 
zero, extra link activation seems to play a bigger role, than 
the choice of the value of M, in reducing the average delays. 

The wireline simulations show the usefulness of the PARN 
algorithm for adaptive routing. However, a wireline network 
does not capture the scheduling aspects inherent to wireless 
networks, which is studied next. 
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Fig. 6. Packet delay as a function of A under PARN in Sprint GMPLS 
network topology. 

2) Wireless Networks: In the case of wireless networks, 
even with extra link activation, to ensure stability even when 
the arrival rates are within the capacity region, we need e > 0. 
We chose e = 0.1 in our simulations due to reasons mentioned 
in Section lYU 
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Fig. 7. Packet delay as a function of A under PARN in the wireless network 
under 2-hop interference model without network coding. 

In Figures [8] and [9] we show the corresponding results for 
the case where both adaptive routing and network coding 
are used. Comparing Figures IT] and [8] we see that, when 
used in conjunction with adaptive routing, network coding 
can increase the capacity region. We make the following 
observation regarding the case M = in Figure |9j in this 
case, no attempt is made to optimize routing in the network. 
As a result, the delay performance is very bad compared to the 
cases with M > (Figure [8]). In other words, network coding 
alone does not increase capacity sufficiently to overcome the 
effects of back-pressure routing. On the other hand, PARN 
with M > harnesses the power of network coding by 
selecting routes appropriately. 

Next, we make the following observation about network 
coding. Comparing Figures [8] and [9] we noticed that at mod- 
erate to high loads (but when the load is within the capacity 
region of the no coding case), network coding increases delays 
slightly. We believe that this is due to fact that packets are 
stored in multiple queues under network coding at each node: 
for each next-hop neighbor, a queue for each previous-hop 
neighbor must be maintained. This seems to result in slower 
convergence of the routing table. 

Finally, we study the performance of the probabilistic split- 
ting algorithm versus the token bucket algorithm. In our simu- 
lations, the token bucket algorithm runs significantly faster, by 
a factor of 2. The reason is that many more calculations are 
needed for the probabilistic splitting algorithm as compared to 
the token bucket algorithm. This may have some implications 



for practice. So, in Figure 10 we compare the delay perfor- 
mance of the two algorithms. As can be seen from the figure, 
the token bucket and probabilistic splitting algorithms result in 
similar performance. Therefore, in practice, the token bucket 
algorithm may be preferable. 
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Fig. 8. Packet delay as a function of A under PARN for M > in the 
wireless network under 2-hop interference model with network coding. 



Fig. 10. Comparison of probabilistic splitting and token bucket algorithms 
under PARN in the wireless network under 2-hop interference model without 
network coding. 
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Fig. 9. Packet delay as a function of A under PARN for M = in the 
wireless network under 2-hop interference model with network coding. 



IX. Conclusion 

The back-pressure algorithm, while being throughput- 
optimal, is not useful in practice for adaptive routing since the 
delay performance can be really bad. In this paper, we have 
presented an algorithm that routes packets on shortest hops 
when possible, and decouples routing and scheduling using a 
probabilistic splitting algorithm built on the concept of shadow 
queues introduced in J6|, 0. By maintaining a probabilistic 
routing table that changes slowly over time, real packets 
do not have to explore long paths to improve throughput, 
this functionality is performed by the shadow "packets." Our 
algorithm also allows extra link activation to reduce delays. 
The algorithm has also been shown to reduce the queueing 
complexity at each node and can be extended to optimally 
trade off between routing and network coding. 

Appendix A 
The Stability of the Network Under PARN 

Our stability result uses the result in [3 1 and relies on the 
fact that the arrival rate on each link is less than the available 
capacity of the link. 



We will now focus on the case of wireless networks without 
network coding. 

All variables in this appendix are assumed to be average 
values in the stationary regime of the corresponding variables 
in the shadow process. Let <jjL- denote the mean shadow traffic 
rate at link (nj) destined to d. Let p, n j and a„(l + e) denote 
the mean service rate of link (nj) and the exogenous shadow 
traffic arrival rate destined to d at node n. Notice that e comes 
from our strategy on shadow traffic. The flow conservation 
equation is as follows: 



L(ln)eC j:(nj)eC 



a^,yn,d£Af. (26) 



The necessary condition on the stability of shadow queues are 
as follows: 



^2 a nj ^ Mr»i- 

deAf 



(27) 



Since we know that the shadow queues are stable under 



the shadow queue algorithm, the expression (27 > should be 
satisfied. 

Now we focus on the real traffic. Suppose the system has 
an equilibrium distribution and let Af„ be the mean arrival 
rate of real traffic at link (nj) destined to d. The splitting 
probabilities are expressed as follows: 

^ " J _ d , where d + n. (28) 

l^fcejV a nk 

Thus, the mean arrival rates at a link satisfy traffic equation: 
\ d nj = a d n P* j + Y, ^ n Pn 3 Mnj)eC,deM, (29) 

l:(ln)eC 

where d ^ n. 

The traffic intensity at link (nj) is expressed as: 
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(30) 



Now we will show p n j < 1 for any link (nj) £ C. Let 
ct^/(1 + e) for every (nj) £ C, and substitute it 
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into expression (f29|). It is easy to check that the candidate where 



solution is valid by using expression (26i. From (27 1, the 
traffic intensity at link (nj) is strictly less than 1 for any link 
{nj) € C : 
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Mnj ,4^ " J (1 + e)fl n j 
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Thus we have shown that the traffic intensity at each link is 
strictly less than 1. 

The wireline network is a special case of a wireless network. 
Substitute the link capacity c n j for \x n j and set e to be zero, 
and stability follows directly. 

The stability of wireless networks with network coding is 
similar to the case of wireless network with no coding. 

Appendix B 

The Back-pressure Algorithm in the Network 

Coding Case 

Given a set of packet arrival rates that lie in the capacity 
region, our goal is to find routes for flows that use as 
few resources as possible. Thus, we formulate the following 
optimization problem for the network coding case. 



min 2_^ ^nj, PP + 2^i ^(™b0 
(nj)<£C (n\ji)ec 
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Let {q n jd} and {qo n d} be the Lagrange multipliers corre- 
sponding to the flow conservation constraints in problem ( [32} . 
Appending the constraints to the objective, we get 
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If the Lagrange multipliers are known, then the optimal fi can 
be found by solving 



max 
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Similar to the update algorithm of g„d in d7), we can derive 
the update algorithm to compute q n jd : 



qnjd[t + 1] — l^njdM + V7 {f 1 nj,pp + ^nj.broad 
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(35) 
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By choosing jj to be the step-size parameter, Mq n jd looks 
very much like a queue update equation. Replacing Mq n jd by 



p n jd, we get (20i-(23i. It can be shown using the results in 
EH, 11241 that the stochastic version of the above equations are 
stable and that the average rates can approximate the solution 



to ( 32 1 arbitrarily closely. 
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